BC-MPI: Running an MPI Application on Multiple Clusters with BeesyCluster Connectivity

A new software package BC-MPI which allows an MPI application to run on several clusters with various MPI implementations is presented. It uses vendor MPI implementations for communication inside clusters and exploits the multithreaded MPI THREAD MULTIPLE mode for handling inter-cluster communication in additional threads of the MPI application. Furthermore, a BC-MPI application can be automatically compiled and started by the BeesyCluster middleware. The latter allows users to manage and use cluster accounts via a single BeesyCluster account and WWW or Web Services. The middleware connects to clusters via SSH and does not require any software installation on the clusters. Results of various latency and bandwidth tests for intra and inter-cluster communication are presented for BC-MPI using OpenMPI and LAM/MPI and Infiniband or TCP within clusters.